Probing the covariance matrix

By drawing an analogy between the logarithm of a probability distribution and a physical potential, it is natural to ask the question, “what is the effect of applying an external force on model parameters?" In Bayesian inference, parameters are frequently estimated as those that maximize the posterior, yielding the maximum a posteriori (MAP) solution, which corresponds to minimizing φ = −log(posterior). The uncertainty in the estimated parameters is typically summarized by the covariance matrix for the posterior distribution, C. I describe a novel approach to estimating specified elements of C in which one adds to φ a term proportional to a force, f , that is hypothetically applied to the parameters. After minimizing the augmented φ, the change in the parameters is proportional to Cf . By selecting the appropriate force, the analyst can estimate the variance in a quantity of special interest, as well as its covariance relative to other quantities. This technique allows one to replace a stochastic MCMC calculation with a deterministic optimization procedure. The usefulness of this technique is demonstrated with a few simple examples, as well as a more complicated one, namely, the uncertainty in edge localization in a tomographic reconstruction of an object’s boundary from two projections.